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DETAILED ACTION 
Response to Arguments 

Applicant's arguments have been fully considered but they are not persuasive. 

-Regarding the applicants argument that "neither Szeliski nor Luken nor Blake teaches, 
discloses or suggests creating a 3D model of a scene comprising geometry information 
describing one or more objects or features contained in the scene" the Examiner disagrees. The 
cited prior art does teach the previously stated limitations (see Szeliski et al. fig. 32, column 20- 
line 63-column 21, line 6, column 29, 45-64, Luken, figs. 7 and 10, column 3, lines 10-18). 

In response to applicant's arguments against the references individually, one cannot show 
nonobviousness by attacking references individually where the rejections are based on 
combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re 
Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or 
described as set forth in section 102 of this title, if the differences between the subject 
matter sought to be patented and the prior art are such that the subject matter as a whole 
would have been obvious at the time the invention was made to a person having ordinary 
skill in the art to which said subject matter pertains. Patentability shall not be negatived 
by the manner in which the invention was made. 

Claims 1-3, 5-10 and 32-35 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Szeliski et al. US patent no. 6,157,747 in view of Luken US patent no. 5,923,334. 

-Regarding claim 1, Szeliski et al. teaches a computerized method for creating a three 
dimensional model from one or more image panoramas (see column 1, lines 6-11, column 4, 
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lines 8-9 and 66-67): receiving one or more image panoramas representing a visual scene and 
having one or more objects (see fig. 6, column 5, lines 51-52); a directional vector indicating an 
orientation of the visual scene with respect to a reference coordinate system (see fig. 15, column 
22, lines 4-9), creating a three dimensional model of the visual scene from the transformed image 
panoramas using the reference coordinate system and comprising geometry information 
describing the one or more objects contained in the scene (see column 5, lines 7-35, column 9, 
lines 4-26), transforming the image panoramas such that the directional vectors are substantially 
aligned relative to the reference coordinate system (see figs. 11 and 15, column 21, lines 25-59, 
column 22, lines 2-35), and aligning the transformed image panoramas to each other (see figs. 1 
and 2B, column 9, lines 29-36). 

Szeliski does not directly teach a method for determining a directional vector for each 
image panorama. However in similar art Luken teaches said method (see figs. 7-10, 14 and 17, 
column 12, lines 35-44). 

Therefore it would have been obvious to one skilled in the art to combine Szeliski with 
Luken, because it would provide an efficient system for generating and viewing three- 
dimensional panoramic images based environment maps, and offer an improved level of 
interactive graphical feedback (see column 3, lines 5-8). 

-Regarding claim 2, Szeliski teaches a method wherein the directional vector is 
determined based, at least in part, on instructions identifying elements of the image panorama 
received from a user (see column 8, lines 30-32, column 27, lines 64-66). 
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-Regarding claim 3, Szeliski teaches a method wherein the instructions from the user 
identify two or more substantially parallel features in the image (see column 20, line 64-column 
2 1 , line 6 and lines 25-34). 

-Regarding claim 5, Szeliski teaches a method wherein the instructions from the user 
identifying a horizon line of the image panorama (see fig 4, column 9, lines 54-62). 

-Regarding claim 6, Szeliski teaches a method wherein the instructions comprise the 
identification of two or more areas of the image (see figs. 6 and 18, column 20, lines 49-50, 
column 27, lines 32-44), each area containing one or more elements and further comprising 
automatically identifying the two elements contained in the two or more areas (see fig. 6, column 
20, lines 49-50, column 32, lines 24-27). 

-Regarding claim 7, Szeliski teaches a method further comprising using edge detection to 
automatically identify two elements (see fig. 32, column 30, lines 13-17). 

-Regarding claim 8, Szeliski teaches a method wherein the image panoramas are aligned 
relative to the reference coordinate system such that the directional vector is at least substantially 
parallel to one axis of the reference coordinate system (see figs. 11 and 15, column 21, lines 25- 
59, column 22, lines 2-35 [it is obvious that as the image rotates into alignment the directional 
vector will be parallel to one axis]). 

-Regarding claim 9, Szeliski teaches a method wherein the image panoramas are aligned 
relative to the reference coordinate system such that the directional vector is at least substantially 
orthogonal to one axis of the reference coordinate system (see figs. 11 and 15, column 21, lines 
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25-59, column 22, lines 2-35 [it is obvious that as the image rotates into alignment the directional 
vector will be orthogonal to one axis]). 

-Regarding claim 10, Szeliski teaches a method wherein the image panoramas are aligned 
according to instructions received from a user (see column 8, lines 30-32, column 25, lines 64- 
66, column 27, lines 64-66). 

-Regarding claim 32, Szeliski et al. teaches a system for creating a three dimensional 
model from one or more image panoramas (see fig. 2B, column 4, lines 66-67, column 9, lines 4- 
26) comprising: means for receiving one or more image panoramas representing a visual scene 
having one or more objects (see column 4, lines 8-13 [captured = received images] and 66-67); 
means for allowing a user to interact with the system (see column 8, lines 30-32, column 25, 
lines 64-66, column 27, lines 64-66); means for aligning the image panoramas relative to each 
other (see figs. 1, 2B, 1 1 and 15, column 9, lines 29-36, column 21, lines 25-59, column 22, lines 
2-35); and means for creating a three dimensional model from the aligned panoramas (see figs. 1 
and 2B, column 9, lines 29-36), the model comprising geometry information describing the one 
or more objects contained in the scene (see column 5, lines 7-35, column 9, lines 4-26) 

Szeliski does not directly teach a method for determining a directional vector for each 
image panorama. However in similar art Luken teaches said method (see figs. 7-10, 14 and 17, 
column 12, lines 35-44). 

Therefore it would have been obvious to one skilled in the art to combine Szeliski with 
Luken, because it would provide an efficient system for generating and viewing three- 
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dimensional panoramic images based environment maps, and offer an improved level of 
interactive graphical feedback (see column 3, lines 5-8). 

-Regarding claim 33, Szeliski teaches a system wherein the input images comprise two- 
dimensional images (see figs. 1 and 2B, [l^k) are two dimensional]). 

-Regarding claim 34, Szeliski teaches a system wherein the input images comprise three- 
dimensional images including geometry information (see fig. 2B, column 5, lines 7-35, column 
9, lines 4-26). 

Regarding claim 35, Szeliski teaches a system wherein the image panoramas are aligned 
according to instructions received from a user (see figs. 1 and 2B, column 8, lines 30-32, column 
25, lines 64-66, column 27, lines 64-66). 

Claims 13-21 and 37 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Szeliski et al. US patent no. 6,157,747 in view of Blank US patent no. 5,469,536.13. 

-Regarding claim 13, Szeliski does not directly teach a method comprising receiving an 
edit to color information associated with the objects of the image. However in similar art Blank 
teaches said method (see column 6, lines 29-47). Therefore it would have been obvious to one 
skilled in the art to combine Szeliski with Blank, because it would enable the user to quickly and 
efficiently modify or enhance the appearance of an image to desired goal (see column 6, lines 
23-28). 

-Regarding claim 14, Szeliski does not directly teach a method comprising receiving an 
edit to alpha information associated with the objects of the image. However in similar art Blank 
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teaches said method (see fig. 11, column 20, lines 61-66). Therefore it would have been obvious 
to one skilled in the art to combine Szeliski with Blank, because it would enable the user to 
quickly and efficiently modify or enhance the appearance of an image to desired goal (see 
column 6, lines 23-28). 

-Regarding claim 15, Szeliski does not directly teach a method comprising receiving an 
edit to depth information associated with the objects of the image. However in similar art Blank 
teaches said method (see column 3, lines 59-67, column 13, lines 8-16). Therefore it would have 
been obvious to one skilled in the art to combine Szeliski with Blank, because it would enable 
the user to quickly and efficiently modify or enhance the appearance of an image to desired goal 
(see column 6, lines 23-28). 

-Regarding claim 16, Szeliski does not directly teach a method comprising receiving an 
edit to geometry information associated with the objects of the image. However in similar art 
Blank teaches said method (see column 47, lines 11-20). Therefore it would have been obvious 
to one skilled in the art to combine Szeliski with Blank, because it would enable the user to 
quickly and efficiently modify or enhance the appearance of an image to desired goal (see 
column 6, lines 23-28). 

-Regarding claim 17, Szeliski does not directly teach a method comprising: providing a 
user with an interactive drawing tool that specifies edits for one or more objects of the image; 
and receiving the edits made by the user using the interactive drawing tool. However in similar 
art Blank teaches said method (see column 21, lines 18-23, column 28, lines 28-35). Therefore it 
would have been obvious to one skilled in the art to combine Szeliski with Blank, because it 
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would enable the user to quickly and efficiently modify or enhance the appearance of an image 
to desired goal (see column 6, lines 23-28). 

-Regarding claims 18 and 37, Szeliski does not directly teach a method wherein the 
interactive drawing tool (editing tool) is one of an extrusion tool, a ground plane tool, a depth 
chisel tool or a non-uniform rational B-spline tool. However in similar art Blank teaches said 
method (see column 8, lines 46-52, column 34, lines 22-27). Therefore it would have been 
obvious to one skilled in the art to combine Szeliski with Blank, because it would enable the user 
to quickly and efficiently modify or enhance the appearance of an image to desired goal (see 
column 6, lines 23-28). 

-Regarding claim 19, Szeliski does not directly teach a method wherein the interactive 
drawing tool specifies a selected value for depth for objects of the image. However in similar art 
Blank teaches said method (see column 21, lines 14-27, column 22, lines 35-62). Therefore it 
would have been obvious to one skilled in the art to combine Szeliski with Blank, because it 
would enable the user to quickly and efficiently modify or enhance the appearance of an image 
to desired goal (see column 6, lines 23-28). 

-Regarding claim 20, Szeliski does not directly teach a method wherein the interactive 
drawing tool incrementally adds to the depth for objects of the image. However in similar art 
Blank teaches said method (see column 34, lines 8-11). Therefore it would have been obvious to 
one skilled in the art to combine Szeliski with Blank, because it would enable the user to quickly 
and efficiently modify or enhance the appearance of an image to desired goal (see column 6, 
lines 23-28, column 48, lines 26-39). 
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-Regarding claim 21, Szeliski does not directly teach a method wherein the interactive 
drawing tool incrementally subtracts from the depth for objects of the image. However in similar 
art Blank teaches said method (see column 34, lines 22-26). Therefore it would have been 
obvious to one skilled in the art to combine Szeliski with Blank, because it would enable the user 
to quickly and efficiently modify or enhance the appearance of an image to desired goal (see 
column 6, lines 23-28, column 48, lines 26-39). 

Claim Rejections - 35 USC § 102 

(b) the invention was patented or described in a printed publication in this or a foreign 
country or in public use or on sale in this country, more than one year prior to the date of 
application for patent in the United States. 

Claims 11,12, 22-28 and 36 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Szeliski et al. US patent no. 6,157,747. 

-Regarding claim 11, Szeliski et al. teaches a computerized method of interactively 
editing objects in a panoramic image (see column 1, lines 6-11, column 4, lines 8-9 and 66-67, 
column 8, lines 30-32, column 27, lines 64-66) comprising: receiving an image panorama 
representing a visual scene (see fig. 3, column 5, lines 46-47, column 9, lines 55-62), the image 
panorama having one or more objects and a point source (see figs. 3, 4 and 6, column 5, lines 46- 
52); creating a three dimensional model of the visual scene using features of the visual scene and 
the point source (see fig. 2B, column 4, lines 66-67, column 9, lines 4-26); receiving an edit to 
one or more of the objects in the panorama (see fig. 1, column 7, lines 11-20); transforming the 
edit relative to a viewpoint defined by the point source (see column 10, lines 27-38, column 11, 
lineslO-21); and projecting the transformed edit onto the objects (see figs 2B-8, column 3, lines 
58-61, column 31, lines 44-51). 
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-Regarding claim 12, Szeliski teaches a three-dimensional model (see figs. 2B, 27, 30 and 
32, column 28, lines 29-31, column 29, lines 58-61), three-dimensional models inherently 
contain one or more of depth information and geometry information. 

-Regarding claim 22, Szeliski et al. teaches a method for projecting texture information 
onto a geometric feature within an image panorama (see fig. 2B, column 3, lines 58-61, column 
27, lines 34-55) comprising: receiving instructions from a user identifying a three-dimensional 
geometric surface within an image panorama (see column 7, lines 10-20), the image panorama 
containing features having one or more textures (see column 5, lines 27-35); determining a 
directional vector from the three-dimensional geometric surface (see figs. 2B and 15, column 22, 
lines 4-31); creating a geometric model of the image panorama based at least in part on the three- 
dimensional geometric surface and the directional vector (see figs. 2A-B, column 7, lines 25-33, 
column 9, lines 4-26, column 22, lines 4-31); and applying the one or more textures to the 
features in the image panorama based on the geometric model (see fig. 2B, column 9, lines 29- 
31, column 32, lines 22-29 and 41-43). 

-Regarding claim 23, Szeliski teaches a method wherein the instructions are received 
using an interactive drawing tool (see column 27, line 64-colum 28, line 2). 

-Regarding claim 24, Szeliski teaches a method wherein the three-dimensional geometric 
surface is one of a floor, a wall, or a ceiling (see figs. 33 and 38, column 30, lines 44-45). 

-Regarding claim 25, Szeliski teaches a method wherein the directional vector is 
orthogonal to the planar surface (see fig. 15, column 15, lines 64-66, column 22, lines 4-6). 
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Regarding claim 26, Szeliski teaches a method wherein the geometric model comprises 
depth information (see figs. 2B, 27 and 30, column 28, lines 29-33). 

-Regarding claim 27, Szeliski teaches a method wherein the texture information 
comprises color information (see column 28, lines 13-18). 

-Regarding claim 28, Szeliski teaches a method comprising environment based texture 
mapping (see fig. 2B, column 5, lines 27-30, column 9, lines 12-26), in said method the texture 
information inherently includes luminance information (see figs. 33-38). 

-Regarding claim 36, Szeliski et al. teaches a system for interactively editing objects in a 
panoramic image (see column 1, lines 6-11, column 4, lines 8-9 and 66-67, column 8, lines 30- 
32, column 27, lines 64-66) comprising: a receiver for receiving one or more image panoramas 
representing a visual scene (see fig. 3, column 5, lines 46-47, column 9, lines 55-62) having one 
or more objects and a point source (see figs. 3, 4 and 6, column 5, lines 46-52); a modeling 
module for creating a three dimensional model of the visual scene (see figs. 2B, 27, 30 and 32, 
column 4, lines 66-67, column 9, lines 4-26, column 28, lines 29-31, column 29, lines 58-61) 
including depth information describing the objects (three-dimensional models inherently contain 
depth information), one or more interactive editing tools for providing an edit to one or more 
objects in the panorama (see fig. 1, column 7, lines 11-20, column 8, lines 30-32); a 
transformation module for transforming the edit relative to a viewpoint defined by the point 
source (see figs. 2B and 8, column 10, lines 27-38, column 11, lineslO-21); and a rendering 
module for projecting the transformed edit onto the objects (see figs. 1 and 2A-B, column 3, 
lines 58-61, column 31, lines 44-51). 
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Conclusion 
THIS ACTION IS MADE FINAL. 

Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A 
shortened statutory period for reply to this final action is set to expire THREE MONTHS from 
the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the 
mailing date of this final action and the advisory action is not mailed until after the end of the 
THREE-MONTH shortened statutory period, then the shortened statutory period will expire on 
the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be 
calculated from the mailing date of the advisory action. In no event, however, will the statutory 
period for reply expire later than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Enrique L Santiago whose telephone number is (571) 272-7648. 
The examiner can normally be reached on Monday to Thursday from 6:30 A.M. to 4:30 P.M. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mark K. Zimmerman whose telephone number is (571) 272-7653, can be reached on 
Monday to Friday from 7:00 A.M. to 3:30 P.M. 

Any response to this action should be mailed to: 

Commissioner of Patents and Trademarks 
Washington, D.C. 20231 

Faxed to: 571-273-8300 

Hand carried deliveries must be made to the Customer Service Window (now located at 
the Randolph Building, 401 Dulany Street, Alexandria, VA 22314) 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
Enrique L. Santiago 

March 16,2007 




KEE M.TUNQ 
SUPERVISORY PATENT EXAMINER 



